chore(ci): add PR Sentinel compliance checks and grace-period auto-close #11254
Open
rafaeltonholo wants to merge 26 commits into
Open
chore(ci): add PR Sentinel compliance checks and grace-period auto-close #11254rafaeltonholo wants to merge 26 commits into
rafaeltonholo wants to merge 26 commits into
Conversation
kryoseu
reviewed
Jul 9, 2026
Member
Author
wmontwe
reviewed
Jul 10, 2026
- Add CI enforcement notice for PR titles and commit subjects - Increase description limit from 50 to 70 characters - Add `perf`, `build`, `ci`, and `deps` commit types
- Add mandatory PR description fields (linked issue, description, AI disclosure, checklist) - Document PR Sentinel automated validation rules and timeline - Clarify PR description guidelines with updated template sections - Add examples using closing keywords for issue linking
…ucture - Add automated PR Sentinel checks notice with compliance requirements and 3-day auto-close timeline - Restructure template sections with better formatting and clearer guidance - Add Testing section for reproducible steps - Improve placeholder text with closing keyword examples - Minor wording improvements in checklist items
- Add Conventional Commits validation for PR titles and commit subjects - Add linked issue check requiring closing keywords - Add AI disclosure section validation - Add commit description length check (max 70 chars) - Add Co-authored-by trailer check - Add bot exemption and escalation logic
- Add tests for PR title, linked issue, and AI disclosure validation - Add tests for commit subject length and Co-authored-by trailer checks - Add tests for bot exemption and escalation logic - Add tests for date helper functions (ISO/epoch conversion) - Add shared assertion helpers for test suites
…tration - Add constants for label, markers, and dry-run mode - Add gh CLI wrapper with dry-run support - Add functions to find, create, update, and delete status comments - Add functions to add and remove PR Sentinel label - Add status comment body renderer with contextual "How to fix" links
- Add evaluate-pr.sh to run all PR Sentinel checks and output JSON verdict - Check PR title, linked issue, AI disclosure, commit subjects, and Co-authored-by trailers - Detect merge commits and flag for rebase - Support bot exemption and draft detection - Add test suite covering compliant/non-compliant PRs, topic filtering, and edge cases
- Add report-pr.sh to evaluate PRs and apply status comment/label - Skip draft PRs and clear prior artifacts when converted to draft - Post/update consolidated status comment for non-compliant PRs - Apply 'pr-sentinel: needs updates' label when non-compliant and not exempt - Exit non-zero for non-compliant, non-exempt PRs to fail CI checks
- Add escalate-pr.sh to escalate or close non-compliant PRs based on status comment age - Escalate after 24 hours by adding close-countdown banner with 3-day deadline - Auto-close after 72 hours if issues remain unresolved - Skip draft PRs and already-escalated comments to avoid duplicate notifications - Add tests for escalation marker detection and age-based decision logic
…rcement - Run on pull_request_target with permissions to comment and label - Support manual dispatch for batch evaluation or dry-run mode - Evaluate PRs for compliance with title, issue link, AI disclosure, and commit requirements - Post consolidated status comments and apply 'pr-sentinel: needs updates' label when non-compliant - Fail CI checks for non-compliant PRs to block merge
…rcement - Run on pull_request_target with permissions to comment and label - Support manual dispatch for batch evaluation or dry-run mode - Evaluate PRs for compliance with title, issue link, AI disclosure, and commit requirements - Post consolidated status comments and apply 'pr-sentinel: needs updates' label when non-compliant - Fail CI checks for non-compliant PRs to block merge
This will avoid the issue 1234 to be linked
The check now ignores closing keywords inside inline code (`Closes thunderbird#1234`) or fenced code blocks, matching GitHub's actual issue-linking behavior where code-enclosed keywords do not create links.
This will allow GitHub to auto-link the commit
…el labels - Add `pr-sentinel: ready for review` label for compliant PRs - Implement mutually exclusive label transitions via mark_ready_for_review, mark_needs_updates, and clear_sentinel_labels - Add tests for label state transitions and mutual exclusivity - Update documentation to reflect new ready-for-review label behaviour
- Add app token generation step with fallback to github.token - Set botmobile environment for both sentinel workflows - Move GH_TOKEN from workflow-level to step-level env vars
rafaeltonholo
force-pushed
the
chore/11158/ci-pr-sentinel
branch
from
July 10, 2026 12:37
6ff232b to
812774f
Compare
wmontwe
reviewed
Jul 10, 2026
wmontwe
left a comment
Member
There was a problem hiding this comment.
This feels a bit over-scoped for the initial Sentinel rollout. That said, shell scripts are acceptable for this scope, and I’d prefer to keep Python usage low in the repository unless there’s a strong reason to add it.
A Kotlin CLI would probably be the preferred option long term for maintainability and testing, but converting this now would delay the rollout and restart a lot of the review.
I’d rather keep this PR as is and then learn how sentinel behaves in real use before deciding whether a Kotlin CLI is a worthy follow-up.
|
|
||
| if [[ $# -ne 4 ]]; then | ||
| echo "Usage: $0 <pr-number> <identifier> <message> <status>" | ||
| if [[ $# -lt 4 || $# -gt 5 ]]; then |
- Move build and ci from commit types to scopes in documentation - Update regex to exclude build and ci as standalone types - Add test coverage for invalid ci type usage
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contribution Summary
Linked Issue/Ticket: Closes #11158
Description
Adds PR Sentinel, an automated bot that validates every pull request against the project's contribution guidelines, gives contributors a single, actionable feedback comment, and enforces the rules with a grace period before any auto-close.
Mandatory checks (per PR):
Closes #123,Part of #123), ignoring keywords inside code so the template placeholder doesn't count.Co-authored-by:trailers.This Pull request also includes:
Testing
To test this PR, you can run the
./scripts/ci/pr-sentinel/tests/all_tests.sh, which will run the script unit tests or run the./scripts/ci/pr-sentinel/report-pr.sh <pr-number> --dry-run.You can also see the PR Sentinel outputs in the following PRs, inside my personal fork:
The above-mentioned PRs will still be open until the PR Sentinel auto-closes them.
AI Disclosure
Select one of the following (mandatory)
Disclosure: Used AI to help with shell script language, script test setup and script organization